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Abstract 

Amethodfor 1 ocal i zati on and posi t i oni ng i n an i ndoor environment is presented. Localizat 
is the act of recognizing the environment, and posi t i oni ng i s the act of computing the e 
coor di nates of a robot i n t he envi ronment . The method is based on representi ng the sc< 
as a set of 2D vi ews and predi cti ng the appear ance of novel views by linear combi nati or 
the model views. The met hod accur at el y approxi nates the appearance of scenes under weak 
perspecti ve project i on. Anal ysi s of t hi s pr oj ecti on as wel 1 as experi ment al resul t s dem 
t hat i n many cases this approxi nati on i s suffici ent to accur at el y descri be the s cene. ^ 
ort hogr aphi c approxi nati on i s i nval id, ei t her a 1 arger nunber of model s can be acqui r ed i 
i t er ati ve sol uti on t o account for the per spect i ve di s tort i ons can be enpl oyed. 

The presented met hod has several advant ages over exi sti ng met hods . It uses rel ati vel y 
represent ati ons , the represent ati ons are 2Dr at her than 3D, and 1 ocal i zati on can be done 
a si ngl e 2D vi ewonl y. The same pri nci pal met hod i s appl i ed bot h f or the 1 ocal i zati on as 
as t he posi ti oni ng probl ent , and a si npl e al gori t hmf or repositioning, t he t ask of r et ur ni 
a previ ousl y vi si t ed posi ti on defined by a si ngl e vi ew, is deri ved f romt hi s method. 
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1 Introduction 

Basi c t asks in aut onomous robot navi gat i on are 1 ocal i zati on and posi t i oni ng. Local i zati or 
act of recogni zi ng t he envi r onnent , t hat is, as si gni ng consi stent 1 abel s t o di fferent 1 ocal 
posit i oni ng i s the act of conputi ng the coor di nates of the robot i n t he envi r onnent . Posi ti 
is a t ask conpl enent ary t o 1 ocal i zati on, i n t he sense that position(e.g., "1.5 meters no 
of t abl e T") is of t en speci fied i n a pi ace- speci fie coor di nat e system( "i n room 9 11") . I 
paper we suggest a met hod of both 1 ocal i zati on and posi ti oni ng usi ng vi si on al one. Aval 
of the posi ti oni ng probl em, referred to as repositioning, i nvol vi ng the ret urn to a pre' 
vi si t ed pi ace i s al so di scus sed. 

Previ ous st udi es have exam ned t he probl ems of 1 ocal i zati on and posi t i oni ng under a va: 
of conditions, defined by the ki nd of sensor(s) employed, the nature of the environment, 
t he represent at i ons used. W can di st i ngui sh between active and pas si ve sensi ng, i ndoo: 
outdoor navi gat i on t asks , and metric and t opol ogi cal representations. The metric appi 
at t e mpt s toutilize adetailedgeometric description of the e nvi r onme nt , whi le the topo 
approach uses a more qual i t ati ve descri pt i on i ncl udi ng a graph wi t h nodes r epresent i ng p 
and arcs represent i ng sequences of acti ons that woul d resul t i n movi ng t he robot f romone ] 
t o another . 

In the paper we consi der a robot that uses a pas si ve sensor , vi si on, i n an i ndoor envi ron 
The envi r onment cannot be changed by t he robot t o i rrprove i t s performance; nei t her beacon 
nor floor or wall markings are employed. The paper addresses both the 1 ocal i zati on and 1 
posi t i oni ng probl ems . Sol uti ons to t hese probl ems are presented based on obj ect recogi 
techniques. The method, based on the linear combinations scheme of [17], represents see 
by sets of their 2D images. Localization is achieved by comparing the observed image 
linear combinations of model views, and the posi ti on of the robot is computed by anal y i 
t he coeffci ent s of t he 1 i near combi nat i on t hat al i gns the model t o t he i mage. Al so, a si ] 
"qualitative" solution to the repositioning problemusing the linear combinations sell 
presented. 

The rest of the paper i s organi zed as f ol 1 ows . The next secti on descri be s the 1 ocal i z at 
posi t i oni ng probl ent and surveys previ ous sol uti ons . The met hod of 1 ocal i zati on and posi t 
usi ng 1 i near combi nati ons of model views is describedinSection3. The method as sumes w 
perspective projection. Alliterative scheme to account for perspective distorti ons i s p 
inSection4. An anal ysi s of t he error r esul ti ng from the proj ecti on assumpt i on i s prese 
Secti on 5. Const r ai nt s i mposed on the moti on of the robot as a resul t of speci al pr oper t 
i ndoor envi r onment s can be used t o reduce t he conpl exi t y of the met hod present ed here. Th 
t opi c i s covered on Sect i on 6. Experi ment al resul t s f ol 1 ow. 



2 The Problem 

Local i z at i on and posi t i oni ng f r omvi sual i nput are denned in the f ol 1 owl ng way: Q ven a 
rri 1 i ar environment, identify the observed environment, and then find your position in 1 
envi r onment . Local i z at i on resembl es the t ask of obj ect recogni t i on, wi t h obj ect s r epl ; 
s cenes . Once 1 ocal i zat i on i s accorrpl i shed, posi ti oni ng can be performed. 

One pr obi em a syst emf or 1 ocal i zat i on and posi t i oni ng shoul d address i s t he vari abi 1 
i mages due to vi ewpoi nt changes . The i nexact ness of pr acti cal systems makes it di ffcul t f 
robot to return to a specified position on subsequent vi si t s . The vi sual dat a avai 1 abl e 
robot between vi si ts vari es i n accordance wi t h t he vi ewi ng posi ti on of the robot . Al ocal 
syst ems houl d be abl e t o recogni ze scenes f romdi fferent posi t i ons and ori ent ati ons . 

Another probl emi s that of changes i n t he scene. M subsequent vi si t s the same pi ace r 
1 ook di fferent due t o changes i n t he arrangement of the obj ect s , t he i nt r oducti on of new obj 
and the removal of other s . In general , some obj ects tendtobe more s t ati c than others . \ 
chai r s and books are of t en moved, t abl es, closets, and pi ct ur es tend to change t hei r po 
much less, and wal Is are al most guaranteed tobestatic. St at ic cues nat ur al 1 y are more re 
t han mobi 1 e ones . Confini ng t he syst em to st ati c cues , however , may i n some cases result 
f ai 1 ure to recogni ze the scene due to i nsuffti ent cues . The syst ems houl d therefore at ten 
r el y on st ati c cues , but shoul d not i gnore t he dynarri c cues . 

Sol ut i ons t o t he pr obi e mof 1 oc al i z at i on f r omvi s ual dat are qui re a 1 ar ge me mor y and h< 
comput ati on. Exi sting systems often try to reduce t hi s cost by usi ng sparse represent ; 
and by expl oi ti ng context ual information. Spar se represent ati ons ar e i nt roduced i n [ 1 
Mit aric [10] represent s scenes as sequences of 1 andmarks (such as wal Is, doors, etc.) ex 
by t r aci ng the boundari es of the s cene usi ng a sonar and a compass . Mt ri c i nf or mat i on 
and between the 1 andmarks i s not stored. Sar achi k [14] recogni zes a room by its dimensi 
whi ch are measured by i dent i fyi ng and 1 ocati ng t he t op corner s of t he roomusi ng stereo ( 
(obt ai ned f romf our earner as ) . In both cases the represent ati on i s very sparse, and t he sc 
t heref or e oft en ambi guous . 

Ei cher represent ati ons are used i n [ 3, 5] where hi gher sue cess rates are reported. Br; 
[3] represent s the scene by an occupancy t abl e, a 2D bi t array whi ch cont ai ns a 1 at e\ 
1 ocati on occupi ed by some obj ect . The t abl e i s const ruct ed by t aki ng stereo pi ctures cc 
360° f romt he rri ddl e of t he room and proj ect i ng t he obt ai ned 3D dat a onto the floor . The 
method suffers f roml oss of i nf or mat i on due to the proj ect i on onto t he floor . 

Engelson et al . [5] represent the scene by a set of invariant "signatures". Asignat 
usual 1 y composed of 1 ow- resol ut i on gray- 1 evel or range dat a obt ai ned by bl urri ng an i mag 
set of si gnat ure s t aken f romdi fferent vi ewpoi nt s are stored. A scene i s recogni zed i f t h 
encounters a si gnat ure si rri 1 ar to one of the stored si gnat ure s . 

Systems t hat use the f ul 1 i nf or mat i on provi ded by t he i mage (e.g., [6, 12]) usual 1 
on contextual i nf or mat i on t o avoi d scanni ng al 1 the model s i n t he memory and to reduce t 
comput ati onal cost of compari ng a model to the i mage. The syst emf ol 1 ows a pr edet errri nc 



path, sot hat t he i denti ty of each vi si t ed 1 ocat i on i s known i n advance, and 1 ocal i zat i on I 
a veri fie at i on probl em Path cont i nui ty i n nany cases is essenti al , and the so- cal 1 ed "drc 
probl emi s not addressed. The errphasi s inthese systems is onpositi oni ng, whi ch i s use 
keep t he robot on the pat h. It i s typi cal for these systems (e.g., [1, 6, 12]) to use a 
model of the environment. 

Qioguchi et al . [ 12] , among ot her s , represent the envi ronment by a set of 1 andmar ks sel e 
f rompai rs of stereoi mages by a human operator . These 1 andmar ks are transformed by an i mag 
processi ng program whi ch i s desi gned so as to i denti fy the speci fie 1 andmar k usi ng spe 
ext r acti on i nst ructi ons (suchas what features tolookfor and at what locations). Local 
is achieved by applying the extraction procedure specified for the next landmark. Qic 
1 andmar k is i denti lied, the posi t i on of the robot rel ati ve to t hat 1 andmar k is det errri n 
compari ng t he di mens i ons of the observed 1 andmar k withthose of the stored model. 

The method presentedin t hi s paper represent s t he envi ronment usi ng a set of edge map 
Local i z ati on and posi t i oni ng are achi eved by compari ng i mages of the envi ronment t o 1 i 
combi nat i ons of the model vi ews . The met hod uses ri ch vi sual i nf or mati on to represent 
scene. The systemis flexible. In many cases it is capable of recognizing its locatio 
one i mage oni y ( 3 6 Ot over age i s not re qui red) . Wen one i mage i s not suffci ent , addi ti ona 
i mages can be acqui red t o sol ve the 1 ocal i zati on probl em Cbnt ext can be used to deterr 
t he order of compari son of the model s t o t he observed i mage and toincreasethe confidence 
a gi ven mat ch, but context i s not essenti al : t he syst emcan al so, by perform ng more ext e: 
computations, solve the "drop- off" problem 

3 Tte Metftxi 

The probl ems of 1 ocal i zati on and obj ect recogni ti on are si ni 1 ar in many ways . Both probl 
require the matching of visual images to stored models, either of the environment or o: 
observed obj ect s . Both probl em face sinilardiffcul ties, such as varyi ng i 1 1 uni nat i on con 
and changes i n appearance due to vi ewpoi nt changes . Si ni 1 ar met hodol ogi es therefore cai 
us e d f or s ol vi ng bot h pr obi e ms . 

Aparticular appl i cati on of an obj ect recogni ti on s cheme, the Linear Combinations ( 
s cheme [ 17] , t o t he probl ems of 1 ocal i zati on and posi ti oni ng i s di scus sed bel ow. The envi r 
is representedinthis schemebya smal 1 set of vi ews obt ai ned f romdi fferent vi ewpoi nt s ai 
t he correspondence bet ween the vi ews . A novel vi ewi s recogni zed by compari ng i t t o 1 i i 
combinations of the stored views. Positioning is achieved by recovering the position 
earner a rel at ive to its positioninthe model views from the coeffci ent s of the al i gni ng 
combi nat i on. In the res t of t hi s secti on we revi ewt he 1 i near combi nati ons approach and des 
i t s appl i cati on to both 1 ocal i zati on and posi t i oni ng. The secti on concl udes wi t h a sol i 
t he probl emof reposi ti oni ng, that is, the probl emof ret urni ng to a previ ousl y vi si t ed 
by "1 ocki ng" i nt o an i mage acqui r ed i n that posi ti on. 



3.1 Localization 

The pr obi emof 1 ocal i zati on i s denned as f ol 1 ows : gi ven P, a 2Di nage of a pi ace, and Ai, a t 
storednodels, find a model 1 MA4 such t hat P nat ches M Local i zati on i s t he recogni ti on 
of a pi ace. It can therefore pot enti al 1 y benefit f romusi ng an obj ect recogni ti on net hod 
A common approach t o handl i ng the problemof recogni ti on from di fferent viewpoints is b 
corrpari ng t he stored no del s to the observed envi ronnent after the vi ewpoi nt is recovered 
conpensat ed f or . This approach, cal 1 ed al i gnment , is used i n a nurrber of studies of ot 
recognition[2, 7, 8, 9, 15, 16] . W appl y t he al i gnnent approach to the probl emof 1 ocal i i 
The syst emdes cri bed bel owuses t he "Li near Combi nati ons" (LC) scheme, whi chwas suggest e< 
by LI 1 nan and Bas r i [17]. 

W begi n wi t h a bri ef revi ewof the LC scheme. LCi s defined as f ol 1 ows . Q ven an i nage, w 
const ruct t wo vi ewvect or s f romt he feature poi nt s i n t he i nage, one cont ai ns the x- coor di 
of the poi nt s , and t he ot her cont ai ns t he y- coor di nates of the poi nt s . An obj ect (i n oui 
t he envi r onment ) is modeledbyaset of suchviews, where t he poi nt s i n these vi ews are or di 
in correspondence. The appearance of a novel vi ew of the object is predicted by appl } 
1 i near conbi nati ons to the s tore d vi ews . The pr edi ct ed appearance is t hen compared wi t h 
actual i mage, and t he obj ect is recogni zed if the two mat ch. The advant age of t hi s met 1 
i s t wof ol d. First, vi ewer- centered re present at i ons are used rat her t han obj ect- center 
namely, models are composed of 2Dviews of the observed scene; second, novel appearances ; 
predictedinasi mpl e and accur at e way (under weak per specti ve proj ect i on) . 

Formal 1 y, gi ven P, a 2Di mage of a scene, and Ai, a set of storednodels, the obj ecti ve : 
find a model M G Ms uch t hat P =£ ?=1 oijMj for some const ant s. @ 1Z. It has been shown 
that this scheme accurately predicts the appearance of rigid objects under weak perspe 
proj ecti on (ort hogr aphi c proj ecti on and seal e) . The 1 i ni t ati ons of t hi s proj ecti on m 
discussedlater inthis paper. 

M>re concretely, 1 e t=^ X{,y{, %) , 1 < i < n, be a set of n object points. Under weak 
perspective projection, the po^i=t( o|i, ^r) of these poi nt s i n t he i mage are gi ven by 

x\ = sr nxi + s r 12 y t +s r 13 z t +t x 

y'i = sr 21 x t +sr 22 y t +sr 23 z t +t y (1) 

where r % j are the component s of a 3 X 3 rot ati on mat ri x, and s is a seal e factor . Eewriting 
i n ve c t or e quat i on f or m we obt ai n 

x' = srnx+sr 12 y +sr 13 z +t x l 

y' = sr 21 x+sr 22 y+sr 23 z+t y l (2) 

where x, y, z,'xy£ Pare the vectors qf, %, %, ^ and y' coor di nat es respectively, and 
1=(1, 1, .. . , 1). Consequently, 

x', ye span{x, y, z, 1} (3) 



or , i n other words', axd y' bel ong to a four- di nensi onal 1 i near subspa l ce(cNbt72.ce that 
z', the vect or of dept h coor di nates of the proj ect ed poi nt s , al so bel ongs to t hi s subspac 
fact is used i n Sect i on 4 bel ow. ) A four- di mensi onal space i s spanned by any four 1 i n 
independent vectors of the space. Two vi ews of the scene supply four such vectors [13, 
Etenot e by xl, yi and x 2 , y> t he 1 ocat i on vectors of t he n poi nt s i n t he t wo i mages ; then t her 
exist coeffci ent^, at, <g, q and b\, b_, b, k such that 

x' = a ixi +a 2 yi +a 3 x 2 +a 4 l 

y' = b ixi +b 2 yi +b 3 x 2 +b 4 1 (4) 

( Nate that the vect of yl ready depends on the other four vect or s . ) Si nee R i s a rot ati o 
mat ri x, thecoeffcients satisfythefoll owi ng t wo quadr ati c const r ai nt s : 

aj+al+al- b\- §- l=2(b 1 b 3 - qa 3 ) rn +2( b 2 b 3 - qa 3 )r 12 

a 1 b 1 +a 2 b 2 +a 3 b 3 +( ai& 3 +a 3 &i) rn +( a 2 b 3 +a 3 b 2 ) r 12 =0 (5) 

To deri ve these const rai nt s t he t r ansf or mat i onbetweenthetwomodel views shoul dbe recove 
Thi s can be done under weak per specti ve usi ng a t hi r d i mage. Al t ernati vel y, the const r; 
can be i gnored, i n wtii ch case the syst emwoul d confuse ri gi d t r ansf or mat i ons wi t h affhe or 
Thi s usual 1 y does not prevent successful 1 ocal i z at i on si nee general 1 y scenes are f ai rl 3 
fromone another. 

A LC scheme for the probl em of 1 ocal i zati on i s as f ol 1 ows : The envi r onment is model 
by a set of images with correspondence between the images. For example, a spot can b 
model ed by two of its cor respondi ng vi ews . The correspondi ng quadr ati c const rai nt s may i 
be stored. Localizationis achievedby recover i ng t he 1 i near combi nat i on t hat al i gns the 
to the observed image. The coeffcients are deterrrined using four model points and the 
correspondi ng i mage poi nt s by sol vi ng a 1 i near set of equati ons . Three poi nt s are suffci < 
deterrrine the coeffcients if the quadratic constraints are also considered. Additional 
may be used to reduce the effect of noi se. 

The LC scheme uses vi ewer- cent ered model s , that is, representations that are corrpot 
of images. It has a number of advantages over methods that build full t hree- di mensi c 
model s to represent the scene. First, by usi ng vi ewer- cent ere d model s that cover rel ati ve] 
t r ansf or mat i ons we avoi d t he need to handl e occl usi ons int he scene. If from some vi ewpo 
the scene appears different because of occlusions we utilize anewmodel for these vi ewpc 
Second, vi ewer-centered mode Is are easier to bui 1 d and t o mai nt ai n than obj ect- centered 
The model s cont ai n onl y i mages and correspondences . By 1 i ni t i ng t he t r ansf or mat i on bet to 
t he model i mages one can find t he correspondence usi ng mot i on met hods . If 1 ar ge porti ons 
t he envi r onment are changed betweenvisits anewmodel can be const ruct ed by si mpl y re pi aci 
old images withnewones. 

One probl emwi t h usi ng t he LC scheme for localizationis due to the weakperspective a 
proxi mat i on. Incontrastwitht he probl emof obj ect recogni t i on, where we can general 1 y as 



t hat obj ect s are smal 1 r el at i ve t o t hei r di st ance from the earner a, in 1 ocal i zati on the 
merit surrounds the robot and perspective distortions cannot be neglected. The 1 i rri t at 
of weak per specti ve model i ng are di s cussed bot h mat he mat i cal 1 y and empi ri cal 1 y i n t he ] 
two secti ons . It i s shown that i n many pr act i cal cases weak per spec ti ve i s suffci ent toe 
accurate localization. The main reason is that the problemof 1 ocal i zati on does not r 
accurate measurement s i n t he ent i re i mage; it onl y re qui res i dent i f yi ng a suffci ent numb' 
spots to guarantee accurate naming. If these spots are relatively close to the center 
i mage, or if the dept h di fferences they create are rel ati vel y smal 1 (as in the case of lo< 
a wal 1 when the 1 i ne of si ght is nearl y perpendi cul ar to the wal 1 ) , t he per specti ve di s' 
are rel at i vel y smal 1 , and the system can i denti f y the scene wi t h hi gh accuracy. Al so, 
rel at ed by a t r ansl ati on par al lei t o t he i mage pi ane forma 1 i near space even when per spe 
di st orti ons are 1 arge. Thi s case and other si mpl i fie ati ons are di s cussed i n Sect i on 6. 

By usi ng weak per specti ve we avoi d st abi 1 i ty pr obi ems that frequent 1 y occur inperspei 
comput ati ons . W can therefore compute the al i gnment coeffci ent s byl ooki ng at a rel ati v 
narrowfiel d of vi ew. The ent ire scheme can be viewedas an accumul at i ve process . Rather t h 
acqui ri ng i mages of the enti re scene and compari ng t hemal 1 t o a f ul 1 scene model (as ii 
we recogni ze t he scene i mage by i mage, spot by spot , unti 1 we accumul ate suffci ent convi nci 
i nf or mat i on t hat i ndi c at e s t he i de nt i t y of t he pi ac e . 

Wen perspective distortions are relatively large and weak perspective is insuffcie 
model the envi ronment , two approaches can be used. Que possibilityis to construct a 1 a 
number of mode Is so as to keep the possible changes between the f ani 1 i ar and t he novel vi < 
smal 1 . Al t ernati vel y, an i t er at i ve comput ati on can be appl i ed t o compensate for these c 
t i ons . Such aniterative methodis describedin Secti on 4. 

3. 2 Posi ti oni ng 

Posi t i oni ng i s the probl em of recover i ng t he exact posi ti on of the robot . Thi s posi ti on 
speci fied i n a fixed coor di nat e sys t emassoci at ed wi t h t he envi ronment (i.e., roomcoor di n; 
or it can be associatedwith some model , in whi ch case 1 ocati on i s expres sedwithrespect 1 
posi t i on from whi chthe model views were acqui red. Inthissectionwediscussan appl i c; 
of the LC s cheme t o t he posi ti oni ng probl em 

The i dea i s the f ol 1 owl ng. W assume a model composed of two i magerEd, -2?! t hei r 
r el at i ve posi ti on is gi ven. Q ven a novel '\ nwgefirfit al i gn t he model wi t h t he i mage 
(i.e., 1 ocal i zati on) . By consi deri ng t he coeffci ent s of t he 1 i near combi nati on the robot ' 
r el at i ve to the model i mages i s recovered. To recover the absol ut e posi ti on of the robot 
r oomt he absol ut e posi t i ons of the model vi ews shoul d al so be provi ded. 

As s urn ng Pi i s obt ai ne d f r omPy a r ot at i on R, t r ans 1 at i on J, ^\f and s c al i ng s , t he 
coor di nates of a poi nt i, n(B, jj) , can be wri 1 1 en as 1 i near combi nati ons of the correspondi n| 
model poi nt s i n t he f ol 1 owl ng way: 

x' = a \X\ -\-a 22/1 +a 3X2 +a 4 



y' = b 1 x 1 +b 2 yi+b 3 x 2 +b 4 (6) 

Subs t i t ut i ng f cjrwe obt ai n 

x' = a i^i +a 22/1 +a 3 ( s qia;i +s ri 2 ?/i +s r^i +i ^) +a 4 

/ = b 1 x 1 +b 2 yi+b 3 (s'Kix 1 +sr 1 2yi+sr 1 3Z 1 +t x ) +b 4 (7) 

and r e ar r angi ng t he s e e quat i ons we obt ai n 

x' = (a 1 +a 3 sr 11 )x 1 +(a2+a 3 sr 1 2)yi+(a 3 sr 13 )z 1 +(a 3 t x +a 4 ) 

y' = (b 1 +b 3 sr 11 )x 1 +(b2+b 3 sr 1 2)yi+(b 3 sr 13 )z 1 +(b 3 t x +a 4: ) (8) 

Usi ng these equati ons we can deri ve al 1 the parameters of t he t r ansf orimti on bet ween the ni 
and the i mage. Assume the i mage i s obt ai ned by a rot at i on U, t r an,s laid sue al i ng.s 
Usi ng the ort honor iml i t y const rai nt we can fir st deri ve t he seal e factor 

si = (a 1 +a 3 sr 11 f +(a 2 +a 3 s r 12 f +(a 3 sr 13 f 

= a\ + a 2 2 +als 2 + 2a 3 s(q L r 11 +a 2^12) (9) 

FromEquati ons (8) and ( 9) , by deri vi ng t he component s of t he t r ansl ati gn wecctaar , t 

obt ai n t he posi ti on of t he robot i n t he i mage rel ati ve to its posi ti on i n the model vi ews 

Ax = a 3 t x -\-a 4 

Ay = b 3 t y +b 4 (10) 

s n s 

Note t hat Az is deri ved f r omt he change i n seal e of the obj ect . The rot ati on mat ri x U betw 
Pi and P' i s gi ven by 

a 4 +a 3 s Tii a 2 + a 3 s r 12 a 3 s r 13 
Mn = u 12 = u 13 = 

Sn S n S n ^11^ 

bi+a 3 sv2i b 2 +a 3 s V22 b 3 sr2 3 ^ > 

u 2 i = u 2 2 = u 23 = 

Sn $n $n 

As was al ready ment i oned, the posi ti on of t he robot i s computed here rel ati ve t o t he posi ti 
t he earner a when the first model i magq, Pas acqui red. Ax and Az represent the moti on of 
t he robot f roraP o P, and t he rest of the parameters represent its 3Dr ot ati on and el evati 
To obt ai n t he r el at i ve posi t i on t he t r ansf or mat i on parameters between the mqdeidvi ews , P 
P2, are re qui red. 

3.3 Repositioning 

An i nt eresti ng vari ant of the positioning problem, referred to as repositioning, is def 
f ol 1 ows . Q ven an i mage, cal 1 ed t he target i mage, posi ti on your s el f i n t he 1 ocat i on f ror 
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t hi s i imge was observed Que way t o sol ve t hi s probl emi s to extract t he exact posi t i on f ror 
whi chthe target i mage was obt ai ned and di r ect t he robot t o that position. Inthis secti 
are interestedin a more qual i t at i ve approach. Under t hi s approach posi ti on i s not corrpi 
Instead, the robot observes the environment and extracts only the direction to the t; 
1 ocat i on. Till i ke t he exact approach, the met hod present ed here does not re qui re the r ecc 
of the t r ansf or mat i on between the model views. 

W assume we are gi ven wi t h a model of the envi ronment together wi t h a t ar get i mage. 
The robot i s al 1 owed tot ake new i mages as it is movi ng towards t he t arget . W assume ; 
hori zont al 1 y movi ng pi at form (In ot her words , we assume three degrees of f reedomr at her 
six; t he r obot i s al 1 owe d t o r ot at e ar ound t he ve r t i c al axi s and t r ans late hor i z ont al 1 
val i di ty of t hi s const rai nt is di scus sed i n Secti on 6. ) Bel owwe gi ve a si mpl e comput ati ( 
det er rri nes a pat h whi ch t errri nat es i n t he t arget 1 ocat i on. At each ti me step the robot acqi 
a newi mage and al i gns it wi t h t he model . By compari ng the al i gnment coeffii ent s wi t h t h< 
coeffcients for the target image the robot deterrrines its next step. The algorithmis di 
i nt o t wo s t ages . Int he first st age the robot fixat es on one i dent i fiabl e poi nt and moves a] 
a ci r cul ar pat h around t he fixat i on poi nt unti 1 t he 1 i ne of si ght to t hi s poi nt coi nci d< 
t he 1 i ne of si ght to the correspondi ng poi nt i n t he t arget i mage. In the second st age the 
advances forwardor retreats backward unti 1 it reaches the target location. 

Q ven a model composed of two i magesj afiid P2, i^ i s obt ai ned f r orn^Fy a rot at i on 
about the Y-axis by an angle a, horizontal t r ang,l atridonrfal e f act or s . Q ven a target 
i mage if, i? i s obt ai ned f r ornlfy a si rri 1 ar rot ati on by an angl e 9, t r ansl,aahdrB cal e 
St. Usi ng Eq. (4) the posi ti on of a t arget pp^htc^sc be expres sed as 

Xf = a i^i -fa 3X2 +a 4 

Vt = b 2Vl (12) 

( The rest of the coeffcients are zero si nee the pi atf ormmoves hori zont al 1 y. ) In fact , the 
c i e nt s are gi ve n by 



a\ 



s t si n(a 



sua 



a 3 = ~ ± — (13) 

s si na 

t x s t si n# 
CI4. = tt — '. 

ssina 

b 2 = s t 

( The de r i vat i on i s gi ve n i n t he Appe ndi x. ) 

At everytime stepthe robot acqui res an i mage and al i gns it wi t h t he above model . Assur 
t hat i mage P'\ s obt ai ned as a resul t of a rot ati on by an angl e (f>, t ra^isalnalts oal^.s 



This problem can be consi der ed as a vari ant of t he horn ng probl em A di scus si on of t he gener al horri ng 
probl em wi t h a "si gnat ur e- bas ed" sol uti on can be found i n[ll]. 



The posi t i on of a poi nj,, (y) is expressed by 

X p = C i^i +C3S2 +c 4 

y P = d 2 yi (14:) 



where the coeffiients are gi ven by 

Cl 



4>) 



sua 

s„si neb . . 

c 3 = JL ~^ (15 

ssma 

C4 — t p 



ssma 



rf 



2 



The step performed by the robot is deterrrinedby 



8 = ^-^ (16) 

C3 «3 

That i s , 

ssinfa — d>) s si n( a — 0) . . 

8 = — — - ^-z — =s sintt cot (/)- cot«) (17) 

si 11(f) sine 

The robot shoul d now move so as to reduce the absol ut e val ue of 8 . The di recti on of moti 
depends on the si gn of a. The robot can deduce the di recti on by movi ng si i ghtl y t o t he s 
and checki ng i f t hi s not i on r esul t s i n an i ncrease or decrease of 8 . The moti on i s defin 
f ol 1 ows . The robot moves t o t he ri ght (or to the left, dependi ng on whi ch di recti on reduce 
by a step Ac. 

A new i mage P n i s now acqui red, and the fixat ed poi nt is locatedinthis i mage. Etenot 
its new posi ti on by. xSi nee the mot i on i s par al lei t o t he i mage pi ane the depth val ues of t 
poi nt i n t he t wo vi ews^, afid P n , are i dent i cal . W now want to rot at e t he earner a so as to 
return the fixated poi nt t o i t s ori gi nal posi ti on. The angl e of rot at i on, /3, can be deduce 
t he equat i on 

x p =x n cos (3 +si n/3 (18) 

Thi s equat i on has two sol uti ons . W chose t he one that counters the t r ansl at i on (namel ; 
t r ans 1 at i on i s to t he r i ght , t he c ane ra shouldrotate to the left), and t hat keeps t he a 
rotation small. In the next time step the new pi n c tape aPes p i^nd the procedure is 
repeated unti 1 8 vani shes . The resul ti ng path is ci r cul ar around t he poi nt of focus . 

Once t he robot arri ves at a posi ti on for whi ch 8 =0 (namel y, its 1 i ne of si ght coi n 
wi t h t hat of the t ar get i mage, and (f> =0) it shoul d now advance forward or retreat backwa: 
t o adj ust its posi t i on al ong the 1 i ne of si ght . Several measures can be used t o det erni 
di recti on of moti on; one exampl e i s t heij/er nsici ch sat i sfies 

^L= 6 JL (19) 

when t he t wj 1 i nes of si ght coi nci de. The obj ect i ve at t hi s st age is t o bri ng t hi s measur< 
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4 Handling Iferspective Distortions 

The 1 i near corrbi nati on scheme presented above accur at el y handl es changes i n vi ewpoi nt ass 
i ng t he i nnges are obt ai ned under weak per spect i ve proj ecti on. Error anal ysi s and experi n 
r esul t s demons t r at e that i n many pr acti cal cases t hi s as surrpti on i s val id. In cases wher 
spect i ve di st orti ons aretoolargetobe handl ed by a weak per spect i ve approxi nati on, nat i 
bet ween t he model and t he i nage can be f aci litated in two ways . Che pos si bi 1 i t y i s to ai 
cases of 1 arge per spect i ve di st orti on by augment i ngthe libraryof storedmodels with add: 
model s . In a rel ati vel y dense 1 i br ary there usual lyexists a model that is relatedtotli 
by a suffci entl y snal 1 trans for nati on avoi di ng such di st orti ons . The second al t ernati ve 
i nprove t he nat chbetweenthe model and the i nage us i ng an i ter ati ve process . In t hi s sect 
we consi der the second opti on. 

The suggested iter at ive process is based on a Tayl or expansi on of t he per spect i ve co 
nates. As described below, this expansion results in a pol ynoni al consisting of term 
of whi ch can be approxi nat ed by 1 i near conbi nat ions of views. The first term of this sei 
r epresent s the ort hogr aphi c approxi nati on. The pr oces s resenbl es a met hod of nat chi ng 
poi nt s wi t h 2Dpoi nts descri bed recent 1 y by EteMnt hon and Eavi s [ 4] . Inthis case, howe\ 
t he met hod i s appl iedto2Dmodels rather than 3D ones . In our appl i cat i on the 3Dcoor di nat 
of the model poi nt s are not pr ovi ded; i nstead they are approxi nat ed fromt he model views. 

An i nage poi nt (x , y) =( f X/ Z, fY/Z) is the proj ecti on of some obj ect poi nt , (X, Y, 
t he i nage, where / denotes the focal 1 engt h. Cbnsi der the f ol 1 owl ng Tayl or expansi on of 
ar ound s one de pt h val ueo^ 



1 
Z 



g/^) (z _ # 



fc=o 



k\ 



E 



t) ( Z - Z) k 



Z ' £{k- l)!z o fc+1 



20' 



1 
Z~ Q 



\ | y (- l fc ) ( Z- Z ^ 

t(k- m z 



The Tayl or series descri bi ng the posi ti on of a poi nt x i s t heref ore gi ven by 

f) I Z - 7U1 



fx = fx 
z z 



i + E 



fc=l 



k - IK Z, 



2V 



Noti ce that the zero t er mcont ai ns the ort hogr aphi c approxi nati on for x . ffirtdtee by A 
fcthtermof the series: 

A ( k) _f_lA^A(^A) k (22) 

" Z (k- 1% Z J {ZZ) 

A r e c ur s i ve de fini tionof the above series is givenbelow. 
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Initialization: 

x (o) =A (o) 



Z 



Iterative step: 



A (fc) = _ Z- Z, (k _ t) 



k - 1 



)o^ 



x (k) = x (^-1) +A (A:) 

where x ' r epresent s the kt h or der appr oxi nati on for a>,> araprA s e nt s the hi ghest order 
t e r mi n x ' . 

Ac cor di ng t o t he or t hogr aphi c appr oxi rrati on both Xand Z can be expressed as 1 i near con 
bi nat i ons of t he model vi ews ( Eq. (4)). W t heref ore appl y the above procedure, approximv 
Xand Z at every step us i ng the 1 i near corrbi nati on that best al i gns the no del poi nt s wi t h 
i nage poi nt s . The general i dea i s therefore the f ol 1 owi ng. Fi r st , '^aaisitiMfi/tbya; 
sol vi ng t he or t hogr aphi c case. Then at each step of t he i t er at i on we i nprove the esti irat 
seeki ng t he 1 i near corrbi nati on t hat best esti nat es the factor 

Z- % _ x- x k ~V 



(k - l)oZ A( fc -!) 

Etenot e by x£ Tt t he vector of i nage poi nt coor di nates , and denote by 

P=[ Xl , H , 3g, 1] (24) 

an n X 4 nat ri x cont ai ni ng the posi ti on of t he poi nt s i n t he two model i nages . Denote 
P + = ( F ' P) _1 P t he pseudo- i nver se of P ( we as sume P i s over deter ni ned) . Al so denot e 
by a'" the coeffcients conputedfor the fcthstep" represents the linear corrbi nati on 
conputedat that st ep t o appr oxi nat e t he Xor the Z values. Since at ev^ry/s,t apdZ 
A; are const ant t hey can be merged i nt o the 1 i near corrbi nati on. BOTofeaicbyA>r) t he 
vectors of conput ed val ues of x and Aat the fcthstep. Alliterative procedure t o al i gn a mo 
t o t he i nage is descri bed bel ow. 

Initialization: 

Solve the orthographic appr oxi nati on, namely 

a(°) = P+x 



Iterati^e step: 



x (o) =A (o) = pa (0) 



q( fc ) = (x- X (^))-A( fc " 1 ) 

a (*) = P + q (A) 

A (k) = ( Pa (fc)) ^ A (fc-i) 

x (fc) = x (fc-i) +A (fc) 
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where the vector operations ® and -^ 



u ® v 
u-^ V 



are denned as 



Ml 



u r 

V r . 



5 Rejection Mdel — Error Analysis 

In t hi s section to esti mate t he error obt ai ned by usi ng t he 1 i near combi nati on method, 
method as sumes a weak per specti ve proj ect i on model . W compare t hi s as sumpti on wi t h the 
more accurate perspecti ve proj ecti on model . 

Apoi nt (X, Y, Z) i s proj ect ed under the perspecti ve model to (i, y) =(fX/Z, fY/Z) i ] 
image, where / denotes the focal length. Under our weak per specti ve model the same poi 
i s approxi mat ed by ( £ , y) = (s X, sY) where s is a s cal i ng f act or . The best esti mate for 
s cal i ng f act or , is gi ven by so^=jv^e2e #i s the average dept h of t he observed envi ronment . 
Denote the error by 

E = \x - k (25) 



The error is expres sed by 



Changi ng t o i mage c oor di nat e s 



or 



E: 



E: 



x Z( 



1 


1 


z~ ~ 


" ~z 


1 


1 



z z 



E: 



z 

z~ 



:. 



26) 



'27) 



'28) 



The error is smal 1 when t he measured feat ure is cl ose the opti cal axi s , or when our esti 
for the depth, o,^is close to the real depth, Z. This supports the basic i nt ui ti on t hal 
i mages wi t h 1 owdept h vari ance and for hxat ed r egi ons ( r egi ons near t he cent er of t he i ma 
t he obt ai ned perspecti ve di st or ti ons are rel ati vel y smal 1 , and the systemcantherefore 
t he scene wi th hi gh accuracy. Fi gures 1 and 2 show the dept h ri&teEoaif/U/nct i on of a; for 
e =10 and 20 pi xel s , and Tabl e 5 shows a number of exampl es for t hi s f unct i on. The al 1 owe 
depth vari ance, Zj,Z\s computed as a f uncti on of x and thetoleratederror, e. For exampl 
a 10 pi xel error toleratedinaheldof viewof up to ±50 pi xe Is is equi val ent to al 1 owl nj 
vari ati ons of 20%. Fr omt hi s di scus si on i t is apparent that when a model is al i gned t o t he 
t he resul t s of t hi s al i gnment shoul d be j udged di ffer entl y at di fferent poi nt s of t he i mage 
farther away a poi nt i s f romt he center the more di scr epancy shoul d be tolerated between 
predi cti on and the actual i mage. Ahve pi xel error at posi ti on x =50 i s equi val ent to a 10 
error at positions =100. 

So far we have consi dered t he di screpanci es between the weak perspecti ve and the per sp 
t i ve proj ecti ons of poi nt s . The accuracy of the LC scheme depends on the val i di ty of the i 
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100 150 200 250 



Fi gur e l:-^- as a f unct i on of a; for e = 10 pi xel s . 




50 100 150 200 250 300 



Fi gur e 2:-^- as a f unct i on of a; for e =20 pi xel s . 
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Lli 

25 

50 

75 

100 



5 



10 



15 



20 



1.2 1.4 1.6 1.8 

1.1 1.2 1.3 1.4 

1.07 1.13 1.2 1.27 

1.05 1.1 1.15 1.2 



Tabl el: AL 1 owe d de pt h r at i-J^ , as a f unc t i on of 
and the error al 1 owed (e , i n pi xel s ) . 



hal f t he wi dt h of t he fiel d consi der ed 



perspective projection bot h i n t he no del vi ews and for the i ncorri ng i nage. Inthe rest of 
secti on we devel op an er ror t ermf or the LC s cheme assurri ng that bot h t he no del vi ews anc 
t he i ncorri ng i nage are obt ai ned by per specti ve proj ecti on. 



The error obt ai ne d by us i ng t he LC s c he ne i s gi ve n by 

E = \x — ax — b-y— c§— d 



29' 



Si nee the scheme accur at el y predi ct s the appearances of poi nt s under weakperspective pr 
t i on, it s ati sfies 

x=ax\— b{y— c&— d (30) 

whe re accented letters represent orthogr aphi c appr oxi nat ions. As s une that inthe t wo mo 
pi c t ur e s t he de pt h r at i os ar e r oughl y e qual : 



r/M 7 



■^02 
^2" 



3r 



( Thi s condi ti on i s s ati shed, for exampl e, when betweenthe two model i mages the earner a or 
t r ansl at es al ong t he i mage pi ane. ) Usi ng t he f act that 



we obt ai n 



E 






[X = [XZo = ,Zo 

z z z x z 



;— by— c§— d 
a ¥t7tt — b "■ 



32' 



Z M 



> Z M 



7 M 

'Z M 



(ciix— b{y— c$) 



7 M 
Z M 



7M 



33' 
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The error therefore depends on two t er he . The first get s srral 1 er as t he i mage poi nt s get cl 
t o t he center of the frame and as the di fference bet ween the dept h r at i os of the no del and ' 
i rrage get s snal 1 er . The second get s srral 1 er as t he t r ansl ati on conponent get s srral 1 er ar 
the model gets close to ort hogr aphi c. 

Fol 1 owl ng t hi s anal ysi s , weak per s pet i ve can be used as a proj ecti on model when t he de 
vari ati ons i n t he s cene are r el at i vel y 1 owand when t he syst emconcent r at es on the center 
of the i mage. W concl ude that , by fixat i ng on di s ti ngui shed part s of the envi ronment , 
1 i near combi nati ons scheme can be used for 1 ocal i zati on and posi t i oni ng. 

6 Iijoosing Constrairts 

Local i zati on and posi t i oni ng re qui re a 1 arge memory and a great deal of on- 1 i ne conput a1 
A large number of models must be stored to enable the robot to navigate and manipulate 
in relatively large and complicated environments. The computational cost of model - i i 
compari son i s hi gh, and if context ( such as path hi st ory) is not avai 1 abl e t he number of rei 
compari sons may get very 1 arge. To reduce t hi s comput ati onal cost a number of const rai nt s r 
be employed. These constraints take advantage of the structure of the robot, the propert 
i ndoor envi ronment s , and t he nat ur al properti es of t he navi gati on t ask. Thi s secti on exa 
some of these constraints. 

Que t hi ng a sys t emmay at tempt t o do i s to bui 1 d t he set of model s so as to reduce t h 
effect of perspective distorti ons i n or der to avoi d perform ng iterative comput ati ons . 
of the environment obtained when the systemlooks relatively deep into the scene usu; 
s ati sf y t hi s condi ti on. Wen per spec ti ve di st orti ons are 1 arge t he syst emmay consi der mo 
subset s of vi ews rel at ed by a t r ansl at i on par al 1 el t o t he i mage pi ane (perpendi cul ar t o 1 
of sight). In this case the depth values of the points are roughly equal across all 
consi der ed, and it can be shown that novel vi ews can be expressed by 1 i near combi nati ons 
two model vi ews even i n t he presence of 1 arge per spec ti ve di st orti ons . Thi s becomes appj 
f romt he fol 1 owl ng deri vati on. J^^(^) , 1 < i < n be a poi nt proj ect ed i n t he i mage 
to ( x, y) =(fJ$/Zi, fY/Zi), and 1 et {(,xf) be the pr oj ect ed poi nt after applying a ri gi d 
t r ans f or mat ion. As s urn ng t h£,t= Z ; we obt ai n 



Z t x' t = r uXi +r 12 Y t +r 13 Z t +t x 
Ziyi = r 2r Xi +r 22Y t +r 23Z; +t y 



M) 



Ei vi di ng by ^Zwe obt ai n 



rnXi +r 12 yt +r 13 +t , 



Zi 
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/ 1 / N 

Vi = r 2iXt +r 2 2y t +r 23 +t y ^r (35) 

Eewri ti ng thi s in vector equati on f ormgi ves 

x' = r n x+r X2 y+r i 3 l+i ^z" 1 

y' = r 2X x+r 22 y + r 23 l+i ^z -1 (36) 

where x, y, x, and y are the vector s o,f, x/, ^, and $ val ues respecti vel y, lis a vector 
of all Is, and 1 zis a vector of ^/values. Consequently, as i n the weak per specti ve case, 
novel vi ews obt ai ned by a t r ansl ati on par al 1 el to the i mage pi ane can be expressed by 1 i 
corrbi nations of four vectors. 

An i ndoor environment usual 1 y provi des the robot with a flat, horizontal support. Con 
quent 1 y, the moti on of t he camera is often const r ai ned t o rot ati on about the vert i cal ( Y 
and to t r ansl ati on i n the XZ- pi ane. Such mot i on has onl y three degrees of f reedomi nst ea 
t he si x degrees of f reedomi n the general case. Under t hi s cons t r ai nt fewer corresponded 
re qui red t o al i gn t he model wi t h t he i mage. For exarrpl e, i n Eq. (4) ( above) the coeffci e 
a 2 =b i =b 3 =b 4 =0. Three poi nt s rather than four are re qui redtodeterrrine the coeffci ent 
by s ol vi ng a 1 i ne ar system Two , ratherthanthree, are re qui r e d i f t he quadr at i c c ons t r ai 
al so consi der ed. Another advant age t o consi deri ng onl y hori zont al moti on i s the fact the 
moti on const rai ns the possi bl e epi pol ar lines between i mages . Thi s fact can be used to 
t he t ask of correspondence seeki ng. 

Obj ect s i n i ndoor envi r onment s somet i mes appear i n roughl y pi anar settings. In parti c 
the relatively static objects tend to be located along walls. Such objects include wi 
shelves, pictures, closets and t abl es . Wen the assumpt i on of ort hogr aphi c proj ect i on i 
(for exampl e, when the robot is r el ati vel y di st ant f romt he wal 1 , or when the 1 i ne of si 
roughl y perpendi cul ar to the wal 1 ) t he t r ansf or mat i on between any t wo vi ews can be descri 
by a 2Daffhe t r ansf or mat i on. The di mensi on of the space of vi ews of t he scene is then re due 
to three (rather than f our ) , and Eq. (4) becomes 

x' = a 1X1 +a 2 yi +a 4 l 

y' = 6 1 x 1 +6 2 y 1 +6 4 l (37) 

(<% =b 3 =0. ) Oil y one viewis therefore suffci ent to model the scene. 

Mast offce- 1 i ke i ndoor envi r onment s are composed of rooms connect ed by corri dors . Nav 
gati ng i n such an envi r onment i nvol ves maneuveri ng through t he corri dors , ent eri ng and exi 
t he rooms . Not al 1 poi nt s i n such an envi r onment are equal 1 y i mport ant . Juncti ons , pi aces ^ 
t he robot faces a number of opti ons for changi ng i t s di recti on, are more i mport ant than c 
places for navigation. In an indoor environment these places include the thresholds of 
and the begi nni ngs and ends of cor ri dor s . Anavi gati on syst em wo ul d therefore tend to s 
more model s for these poi nt s than for others . 

One i mport ant property shared by many j uncti ons i s t hat t hey are confined to r el at i ve 
smal 1 areas . Consi der for exampl e t he t hreshol d of a room It is a r el ati vel y narrow 
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that separates the roomf romt he adjacent corridor. Wen a robot is about to enter a roc 
a common behavi or i ncl udes s t eppi ng t hrough the door , 1 ooki ng i nt o t he room, and i denti f 3 
it before adecisionis made to enter the roomor to avoi d i t . The set of i nt eres ti ng i ma£ 
t hi s t ask i ncl udes the set of vi ews of t he roomf romi t s ent ranee. Provi ded that t hreshol 
narrow these vi ews are r el at ed to each other al most exel usi vel y by rot ati on around the ve: 
axi s . Under per specti ve proj ect i on, such a rot ati on i s r el at i vel y easy to recover . The 
of points i n novel views can be recovered fromone model view only. This is apparent fr 
t he f ol 1 owl ng deri vati on. Consi der a poi nt p = ( A, Y, Z) . Its posi ti on i n a model vi ewi t 
by (a;, y) =(fX/Z, fY/Z). Now, consider another vi ewobt ai ned by a r ot at i on R around t 
earner a. The 1 oc ati on of p i n t he new vi ewi s gi ven by (assuni ng / =1) 



i npl yi ng t hat 



, ,s _ rnX+r 12 Y +r 13 Z r 2 iX+r 22Y +r 23 Z , N 

• ' ~ ( r 31 X+r 32 Y +r 33 Z'r 31 X+r 32 Y+r 33 Z> ( ' 



, , rnx +r 12 y +r 13 r 21 x +r 22 y +r 23 /0 „s 

r, y) = ( , ) (39 

r 3 ix +r 32 y +r 33 r 31 x +r 32 y +r 33 



Etept h i s t heref ore not a f act or i n det errri ni ng t he rel ati on bet ween t he vi ews . Eq. ( 39) be< 
even si npl er i f onl y rot ati ons about t he Y- axi s are consi dered: 

.... . icosa+sina y . 

K£= — ^ — > ^— : — 40 

— x sma +cos ct- x sma +cos a 

where a i s t he angl e of rot ati on. In t hi s case a can be recovered merely froma si ngl e c 
spondence. 

7 Bcperinarts 

The LC met hod was i npl ement ed and appl i ed t o i mages t aken i n an i ndoor envi r onment . 
Images of two offces , Aand B, t hat have sinilar structures weret aken usi ng a Panasoni c camei 
withafocal 1 engt h of 700 pi xe Is. Seni-staticobjects, suchas heavy f urni t ure and pi ct ur 
used t o di sti ngui sh between the offces . Fi gure 3 shows two model vi ews of offce A The view 
were taken at a distance of about 4mfromthe wall. Cor respondences were pi eked manual 1 
The resul t s of al i gni ng the model vi ews t o i mages of t he two offces are present ed i n Fi gur 
The 1 ef t i mage cont ai ns an overl ay of a predi ct ed i mage (t he t hi ck whi t e 1 i nes ) , const rue 
1 i nearl y combi ni ng t he t wo vi ews , and an actual i mage of offce A A good mat ch between the 
two was achi eved. The ri ght i mage cont ai ns an overl ay of a predi ct ed i mage constructed fi 
a model of offce B and an i mage of offce A Because the offces share a si ni 1 ar structure the 
static cues (the wall corners) were perfectly aligned. The seni-static cues, however, 
mat ch any feat ure s i n t he i mage. 

Fi gur e 5 s hows t he mat chi ng of t he model of offce A wi t h an i mage of t he s ame offce ob- 
t ai ne d by a r el at i vel y 1 ar ge mot i on f or war d ( about 2m) and t o t he s i de ( about 1 . 5m) . M t ho 
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Fi gure 3: Two model vi ews of offfce A 





Fi gure 4: Mt chi ng a model of offfce At o an i mage of offfce A(l ef t ) , and mat chi ng a model of 
offfce B t o t he s ame i mage ( r i ght ) . 




Fi gure 5: Mt chi ng a model of offfce At o an i mage of the s ame offfce obt ai ned by a rel ati vel y 

1 ar ge mot i on f or war d and to t he r i ght . 
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Fi gur e 6: Two model vi ews of a corri dor . 





Fi gur e 7: Mt chi ng the corri dor model wi t h t wo i mages of t he corri dor . The ri ght i mage v 
obt ai ned by a r el at i vel y 1 ar ge mot i on forward ( about hal f of t he corri dor 1 engt h) and t 
ri ght . 

t he di st ances are r el at i vel y short most per specti ve di s t ort i ons are negl i gi bl e, and a go 
between t he model and the i mage i s obt ai ned. 

Another set of images was taken in a corridor. Here, because of the deep structure 
the corridor, perspective distortions are noticeable. Nevertheless, the alignment re; 
demonstrate an accur at e mat ch i n 1 arge por ti ons of t he i mage. Fi gure 6 shows two model vi e 
of the cor ri dor . Fi gure 7 (left) shows an overl ay of a 1 i near combi nati on of t he model 
wi t h an i mage of the corri dor . It can be seen that the part s that are r el at i vel y di s t am 
perfectly. Fi gure 7 (ri ght ) shows the mat chi ng of the cor ri dor model wi t h an i mage obt ai ne 
a r el at i vel y 1 arge moti on (about hal f of the cor ri dor 1 engt h) . Because of per specti ve di i 
t he r el at i vel y near features no 1 onger al i gn (e. g. , the near door edges ) . The rel ati vel y 
however, still match. 

The ne xt e xpe r i me nt s ho ws the appl icationof the iterative process presentedin Sect: 
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in cases where 1 arge per spec ti ve di st orti on were noti ceabl e. Fi gure 8 shows two no del v 
and Figure 9 shows the results of matching a linear combination of the model views to 
i nage of t he sane offie. In t hi s case, because the i nage was t aken froma relatively cl 
di st ance, per spec ti ve di st orti ons cannot be negl ect ed. The effects of perspective di st or~ 
be noti ced on t he ri ght corner of t he board, and on the edges of t he hanger on the top ri i 
Perspective effects were reduced by using the iterative process. The results of appl yi i 
procedure af t er one and three i t er ati ons are shown i n Fi gure 10. 

The experi nent al r esul t s demons t rat e that the LC net hod ac hi eves accurate 1 ocal i zati o 
many cases , and that when the met hod f ai 1 s because of 1 arge per spec ti ve di st orti ons an i t e 
comput at i on can be used to i mprove the qual i t y of the mat ch. 

8 GhncliEions 

A met hod of 1 ocal i zati on and posi ti oni ng i n an i ndoor envi ronment was presented. The met h 
i s based on representi ng the scene as a set of 2Dvi ews and predi cti ng the appearance of r 
views by linear combinations of the model views. The met hod accur at el y appr oxi mat es t. 
appearances of scenes under weak per specti ve projection. Analysis of this projection, 
as experimental results demonstrate that in many cases this approximation is suffcienl 
accur at el y descri be the scene. Wen t he weak per specti ve appr oxi mati on i s i nval id, ei t 
1 arger number of model s can be acqui red or an i t er ati ve sol uti on can be empl oyed to acco 
for the perspective distortions. 

The method presentedin this paper has several advant ages over exi sti ng met hods . It u 
rel ati vel y ri ch represent ati ons ; the represent at i ons are 2Dr at her t han 3D, and 1 ocal i za 
be done froma si ngl e 2D vi ewonl y. The s ame basi c met hod i s used i n both the 1 ocal i zati 
and posi t i oni ng pr obi ems , and a si mpl e al gori t hmf or reposi t i oni ng i s deri ved f romt hi s m 
Future work i ncl udes handl i ng t he probl emof acqui si ti on and mai nt e nance of model s , devel 
i ng effii ent and robust al gori t hms for sol vi ng t he correspondence probl em, and bui 1 di ng r 
us i ng vi s ual i nput . 

^perrix 

In t hi s appendi x we deri ve the expl i ci t val ues of the coeffii ent s of the 1 i near combi nati ons 
case of hori zont al moti on. Consi der a poi nt p =(x , y , z) that is pr oj ect ed by weak per sp 
to three i mages i,Pp, and P, pis obt ai ned f r ornI?y a rot at i on about the F- axi s by an 
angl e a, t r ansl ati^o^i ind s cal e f act oj, sand P i s obt ai ned f r oral? rot ati on about the 
Y- axi s by an angl e 9 , t r ansl ^taroalirfcal e^s The posi ti on of p i n t he t hree i mages is gi ven 

by 

Oi, u) = {x, y) 
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Figure 8: Two model views of offce C. 




Fi gur e 9: Mt c hi ng the model t o an i mage obt ai ned by a rel ati vel y 1 arge moti on. Per spec t 
di st orti ons can be seen i n the t abl e, t he boar d, and t he hanger at the upper ri ght . 





Fi gur e 10: The resul t s of appl yi ng the i t er ati ve process to reduce per spect i ve di st orti 
one (left) and three (right) iterations. 
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(%2, U) = (smicosa+^zsina+fe, ^y) 
( x 1 , y) = ( s p a; cos 9 +fz si n # +j^, #2/ ) 

The poi nt ('3 $) can be expres sed by a 1 i near conbi nat i on of the first t wo poi nt s : 

x' = a \X\ -\-a 2^2 +a 3 
y' = by-L 

Eewri ti ng these equati ons we get 

s p x cos 9 -\-fyz si n9 +pt = flii +fl2(%i ;tcos o+^z sina -\~tn) + a 3 
s p y = by 

Equati ng the val ues for the coeffci ent s i n bot h si des of these equati ons we obt ai n 

s p cos 9 = (\ -\-a 2S m cos a 
Spsi n9 = qs m sina 

t p = a 2t m +0, 3 

s p = b 

and the coeffci ent s are t heref ore gi ven by 

s p si n(a — 9) 



a 3 



sua 



Sp si nl 



s, m sma 



CI4. — t p 



TYl 1} 



s m sma 
b = Sp 

References 

[ f] N. Ayache and 0. D. Faugeras. Mi nt ai ni ng represent ati ons of t he envi ronnent of a nobi 
robot, IEEE Trans, on Robotics and Automat i on, \fol . 5, pp. 804-8f9, 1989. 

[2] R Basri and S. XI 1 nan. The alignment of objects wi th snoot h surf aces . Proc. 2nd I 
Conf. on Cbrrput er Vision, Tarpon Spri ngs , FL, pp. 482-488, 1988. 

[3] D. J. Braunegg. Mrvel — Asyst emf or recogni zi ng worl d 1 ocati ons wi t h s t ereo vi si on 
TR-1229, MIT, 1990. 

[4] D. F. EteMnt hon and L. S. Davis. M>del - based obj ect pose i n 25 1 i nes of code. Proc. 2 
European Cbnj . onCbrrputer Vision, Genova, Italy, 1992. 

22 



[5 



[7 



10 



11 



12 



13 



14 



15 



16 



17 



S. P. Engelson and D. V. MLtermott. Iimge signatures for place recognition and ma 
construction. Proc. SPIE Syrrposi union Intel I i gent Boboti c Systems, Boston, M\, 1991. 

C. Fennena, A Hanson, E. Eisenan, R J. Beveridge, and R Kunar . M>del - di rect ed 
mobile robot navigation. IEEE Titans, on Systems, Mn and Cybernetics, \o\ . 20, pp. 1352- 
1369, 1990. 

M A Fischler and R C. Bolles. Randoms arrpl e consensus : a par adi gmf or model fit t i i 
with application to image analysis and automated cartography. Cbmruni cat i ons of the 
ACA{ Vol. 24, pp. 381-395, 1981. 

D. R Hut t enl ocher and S. LI 1 man. Obj ect recogni ti on usi ng al i gnment . Proc. 1st Int. C 
on Computer Vision, London, UK, pp. 102-111, 1987. 

D. G Lowe. Three-dimensional object recognition from single two-dimensional i me 
Boboti cs Besearch Technical Beport 202, Gourant Institute of Mth. Sciences, New York 
University, 1985. 

M J. Mtaric. Envi ronment 1 earni ng usi ng a di stri but ed represent ati on. Proc. Int. 
on Boboti cs and Autonati on, Qncinnati, OH, 1990. 

R N. Nelson. Visual honing using an associative memory. DABPAInnge Understanding 
Workshop, pp. 245-262, 1989. 

K Qioguchi , M Wtanabe, Y. Okamoto, Y. Kuno, and H Aada. A visual navigation 
syst emusi ng a mil t i i nf ormati on 1 ocal map. Proc. Int. Gonf . on Boboti cs and Automation, 
Cincinnati, OH, pp. 767-774, 1990. 

T. Poggi o. 3D obj ect recognition: on a result by Basri and LI 1 man. Technical Bspc 
9005-03, IBST, Povo, Italy, 1990. 

K B. Sarachik. Visual navigation: const ructi ng and uti 1 i zi ng si mpl e maps of an i 
environment. AI- IB- 1113, AIT, 1989. 

D. W Thompson and J. L. Mmdy. Three dimensional model matching froman uncon- 
strained vi ewpoi nt . Proc. Int. Gonf. on Boboti cs and Automation, Raleigh, NC, pp. 208- 
220, 1987. 

S. LI 1 man. M i gni ng pi ct ori al descriptions: an approach to obj ect recognition. Go 
Vol. 32, pp. 193-254, 1989. 

S. LI 1 man and R Basri . Recogni ti on by 1 i near combi nati ons of model s . IEEE Trans, c 
Pattern Analysis and Mchine Intelligence, \bl . 13, pp. 992-1006, 1991. 



23 



